Synthetic multi-string musical instrument with score coded performance effect cues and/or chord sounding gesture capture

ABSTRACT

A synthetic multi-string musical instrument captures a stream of expressive gestures indicated on a multi-touch sensitive display for note/chord soundings and associated performance effects and embellishments. Visual cues in accord with a musical score may be revealed/advanced at a current performance tempo, but it is the user&#39;s gestures that actually drive the audible performance rendering via digital synthesis. Opportunities for user expression (or variance from score) include onset and duration of note soundings, tempo changes, as well as uncued string bend effects, vibrato, etc. Gesturing mechanism are provide to allow user musicians to sound chords without having to register precisely accurate multi-touch screen contacts. This can be especially helpful for mobile phone, media player and game controller embodiments, where there is generally limited real-estate to display six (6) or more strings, and user fingers are generally too fat to precisely contact such strings.

CROSS-REFERENCE TO RELATED APPLICATION(S)

The present application claims the benefit of U.S. ProvisionalApplication No. 61/570,701, filed Dec. 14, 2011, the entirety of whichis incorporated herein by reference.

BACKGROUND

1. Field of the Invention

The invention relates generally to musical instruments and, inparticular, to techniques suitable for use in portable device hostedimplementations of musical instruments for capture and rendering ofmusical performances with game-play features.

2. Related Art

The installed base of mobile phones and other handheld compute devicesgrows in sheer number and computational power each day. Hyper-ubiquitousand deeply entrenched in the lifestyles of people around the world, theytranscend nearly every cultural and economic barrier. Computationally,the mobile phones of today offer speed and storage capabilitiescomparable to desktop computers from less than ten years ago, renderingthem surprisingly suitable for real-time sound synthesis and otherdigital signal processing. Indeed, modern mobile phones and handheldcompute devices, including iOS™ devices such as the iPhone™, iPod Touch™and iPad™ digital devices available from Apple Inc. as well ascompetitive devices that run the Android operating system, tend tosupport audio (and indeed video) playback and processing quite capably.In addition, multi-touch user interface frameworks provided in suchdevices create new opportunities for human machine interactions.

These capabilities (including processor, memory and I/O facilitiessuitable for real-time digital signal processing, hardware and softwareCODECs, audiovisual and touch screen APIs, etc.) have contributed tovibrant application and developer ecosystems. Examples from thesynthetic musical instrument application space include the popularOcarina, Magic Piano and Magic Guitar social music apps and theinnovative Magic Fiddle app, all from Smule, Inc. As synthetic musicalinstruments designs develop and mature, innovative techniques are neededto simulate, extend and, indeed, improve upon musician-instrumentinteractions that, while familiar from the physical world, presentchallenges for implementations of synthetic instruments on generalpurpose hardware such as the ubiquitous mobile phones and handheldcompute devices discussed above. Likewise, innovative digital syntheticinstrument designs are needed to tap the potential of engaging social,interactive, and even game-play experiences.

SUMMARY

Despite practical limitations imposed by mobile device platforms andapplications, truly captivating musical instruments may be synthesizedin ways that allow musically expressive performances to be captured andrendered in real-time. In some cases, synthetic musical instruments canprovide a game, grading or instructional mode in which one or morequalities of a user's performance are assessed relative to a musicalscore. In some cases, embodiments or modes, visual cues presented on amulti-touch sensitive display provide the user with temporally sequencednote and/or chord selections throughout a performance in accordance withthe musical score. Note or chord soundings (or selections) are indicatedby user gestures captured at the multi-touch sensitive display, and oneor more measures of correspondence between actual note soundings and thetemporally sequenced note and chord selections may be used to grade theuser's performance. In some cases, embodiments or modes, note soundingsmay be coordinated but gestured (and indeed graded) separately fromnote/chord selections.

In general, particular visual cuing and note sounding gesture sets maybe particular to the synthetic musical instrument implemented. Forexample, in a piano-type synthetic instruments configurationsreminiscent of that popularized by Smule, Inc. in its Magic Pianoapplication for iPad devices, user digit contacts (e.g., finger and/orthumb contacts) at laterally displaced positions on the multi-touchsensitive display constitute gestures indicative of key strikes, and adigital acoustic model of a piano is used to render an audibleperformance in correspondence with captured user gestures. In contrast,synthetic multi-string instruments such as that popularized by Smule,Inc. in its Magic Fiddle application for iPad devices typically presenta visual depiction of a multiplicity of strings with which a user mayinteract to sound notes and/or chords.

It has been discovered that, notwithstanding the multi-string nature ofcertain instruments that may be synthesized (e.g., guitars), alternativevisual cuing and gesture expression models may be more suitable for thelimited touch-screen real-estate provided by mobile handheld devicessuch as phones, media players, gaming controllers, etc. Accordingly, a“piano roll” style set of visual cues has been adopted to provide asong-mode user of a synthetic guitar-type (multi-string) musicalinstrument with note and/or chord selection cues (in accordance with amusical score). Thus, the visual cuing and note expression gesture modelfor a multi-string fretted instrument can rationalized and simplified toa laterally extended note sounding zone, without regard to frettedquantization of pitches or allocations to respective strings.

The synthetic guitar-type (multi-string) musical instrument captures astream of expressive gestures indicated on a multi-touch sensitivedisplay for note/chord soundings and associated performance effects andembellishments. While the visual cues are driven by a musical score andrevealed/advanced at a current performance tempo, it is the user'sgestures that actually drive the audible performance rendering.Opportunities for user expression (or variance from score) include onsetand duration of note soundings, tempo changes, as well as uncued stringbend effects, vibrato, etc.

It has further been discovered that, by using a fretless, laterallyextended note sounding zone, lateral extent can be dynamically allocatedso that the pitch range of available notes (and spacing between noteselective positions) can be allocated to laterally distribute a currentcontextually relevant set of note sounding positions along the availablelateral extent of a multi-touch sensitive display. In general, rangeand/or spacing may be dynamically selected based a note set employed ina current song selection or portion thereof. In this way, visual cuesand user note sounding gestures may be distributed across themulti-touch sensitive display in a manner that makes it easier for theuser musician to play. The synthetic guitar may allocate finger space toeach note, but need only allocate space for the note set that isnecessary to play the current selection or song (or to play at a currentpoint within a current selection or song).

It has further been discovered that, particularly in freestyle modes ofplay, it is possible to provide users with a gesturing mechanism tosound chords without having to register precisely accurate multi-touchscreen contacts. This is especially important for mobile phone, mediaplayer and game controller embodiments, where there is generally limitedreal-estate to display six (6) or more strings, and user fingers aregenerally too fat to precisely contact such strings. Instead, for agiven key and context, most of the important chords have adistinguishing “shape” and location in terms of the fingers that contactthe strings and strings that are contacted.

By recognizing in multi-touch gesture capture, that an E major chord isgestured as an obtuse triangle of finger contacts at the top of thestrings (screen), an A minor chord has the same shape but is lower onthe screen, an A major chord is gestured as a linear arrangement offinger contacts near the bottom of the screen, a C major chord isgestured as an upward diagonal arrangement of finger contacts at thecenter of the screen, and a D major chord is gestured as a nearlyequilateral triangle of finger contacts at the bottom of the screen,etc., it is possible to reliably guess at the most likely chord soundingexpressed by the user. The user doesn't have to hit strings and fretsexactly. Indeed, positional registration along respective string displayartifacts need not matter at all, or can a little but not as criticallyas real frets. Rather, the user may simply gesture finger contacts inroughly the right shape at roughly the right portion of the screen.Gesture recognition and capture algorithm(s) identify the sounded chordand supply appropriate gesture-driven inputs to a multi-stringsynthesis. In some embodiments, strings are displayed or dynamicallyrevealed beneath the finger contacts in correspondence with therecognized chord to visually reinforce the chord indication.

In general, audible rendering includes synthesis of tones, overtones,harmonics, perturbations and amplitudes and other performancecharacteristics based on the captured gesture stream. In some cases,rendering of the performance includes audible rendering by converting toacoustic energy a signal synthesized from the gesture stream encoding(e.g., by driving a speaker). In some cases, the audible rendering is onthe very device on which the musical performance is captured. In somecases, the gesture stream encoding is conveyed to a remote devicewhereupon audible rendering converts a synthesized signal to acousticenergy.

Thus, in some embodiments, a synthetic musical instrument (such as asynthetic guitar) allows the human user to control an actual expressivephysical model of a vibrating string using multi-sensor interactions(e.g., fingers on strings or at frets, strumming, bending, damping,etc.) via a multi-touch sensitive display. In this way, the user isactually causing the sound and controlling the parameters affectingpitch, quality, etc. A variety of computational techniques may beemployed and will be appreciated by persons of ordinary skill in theart. For example, exemplary techniques include (i) wavetable or FMsynthesis and, in some cases, embodiments or modes (ii) physicalmodeling of string vibration and acoustics of the modeled instrument.

In some embodiments, a storybook mode provides lesson plans which teachthe user to play the synthetic instrument and exercise. Userperformances may be graded (or scored) as part of a game (orsocial-competitive application framework), and/or as a proficiencymeasure for advancement from one stage of a lesson plan to the next. Ingeneral, better performance lets the player (or pupil) advance faster.High scores both encourage the pupil (user) and allow the system to knowhow quickly to advance the user to the next level and, in some cases,along which game or instructive pathway. In each case, the user isplaying a real/virtual physical model of an instrument, and theirgestures actually control the sound, timing, etc.

Often, both the device on which a performance is captured and that onwhich the corresponding gesture stream encoding is rendered areportable, even handheld devices, such as pads, mobile phones, personaldigital assistants, smart phones, media players, or book readers. Insome cases, rendering is to a conventional audio encoding such as AAC,MP3, etc. In some cases, rendering to an audio encoding format isperformed on a computational system with substantial processing andstorage facilities, such as a server on which appropriate CODECs mayoperate and from which content may thereafter be served. Often, the samegesture stream encoding of a performance may (i) support local audiblerendering on the capture device, (ii) be transmitted for audiblerendering on one or more remote devices that execute a digital synthesisof the musical instrument and/or (iii) be rendered to an audio encodingformat to support conventional streaming or download.

Chord Sounding Gesture Capture

In some embodiments in accordance with the present invention(s), amethod of providing a synthetic instance of a multi-string musicalinstrument using a portable computing device includes (i) capturingchord sounding gestures indicated by a user on a multi-touch sensitivedisplay of the portable computing device, (ii) distinguishing betweenrespective multiple-finger contact geometries gestured by the user and,based thereon, passing a stream of chord indicative inputs to a digitalsynthesis of the multi-string musical instrument executing on theportable computing device, and (iii) in correspondence with the streamof captured chord indicative gestures, audibly rendering a performanceon the portable computing device using the digital synthesis. Thecaptured chord sounding gestures include, for particular and respectivechords so gestured, contact by multiple fingers of the user's hand andwherein the captured chord sounding gestures are characterized byrespective geometries of the multiple-finger contacts rather than solelyby positional registrations of individual finger contacts relative tostring or fret positions.

In some embodiments, the chord sounding gesture capturing includesretrieving, in response to a multi-touch application programmerinterface (API) notification, a touch count and coordinates forrespective ones of the touches. In some cases, the distinguishingincludes, for two-touch contact gestures indicative of a two-note chordcharacterizing orientation of line traceable through the touchcoordinates and based thereon distinguishing between at least somesupported ones of the two-touch contact gestures. In some cases, thedistinguishing further includes characterizing first-dimensioncoordinates of a captured two-touch contact gesture to distinguishbetween major and minor chord. In some cases, the distinguishingincludes, for three-touch contact gestures indicative of a three-notechord, characterizing both shape defined by the touch coordinates and anorientation thereof and, based on the shape and orientationcharacterization, distinguishing between at least some supported ones ofthe three-touch contact gestures.

In some cases, the distinguishing further includes (i) for a three-touchcontact gesture characterized as an upward-pointed, triangulararrangement, indicating a G major chord; (ii) for a three-touch contactgesture characterized as an downward-pointed, triangular arrangement,indicating a D major chord; (iii) for a three-touch contact gesturecharacterized as a generally-linear arrangement, sloping upwardsleft-to-right, indicating a C major chord, and (iv) for a three-touchcontact gesture characterized as a generally-linear arrangement, slopingdownwards left-to-right, indicating an F major chord.

In some cases, the distinguishing further includes for four-touchcontact gestures indicative of a four-note chord (i) characterizing bothshape defined by coordinates of an initial three of the four touches andby an orientation thereof; (ii) further characterizing a positionalrelation between the characterized shape and a fourth of the fourtouches; (iii) based on the shape and orientation characterization ofthe initial three (3) touch coordinates, initially distinguishingbetween at least some supported major chords; and (iv) based on thecharacterized positional relation, distinguishing between major anddominant 7^(th) variants of the initially distinguished major chords.

In some cases, the distinguishing includes characterizingsecond-dimension coordinates of a captured multi-touch contact gestureto distinguish between a root chord and one or more successiveinversions thereof. In some cases, the chord sounding gesture capturingincludes capture of both: a multi-touch chord selection gesture; and atraveling-touch, strum-type gesture.

In some embodiments, the method includes providing a synthetic strum inaccord with score-coded meter, the synthetic strum releasing aconstituent note or chord indications to the digital synthesis based ona then-current one of the captured note sounding gestures.

In some cases, the distinguishing is contextually constrained based atleast in part on a then-current musical key. In some cases, the currentmusical key is either user-selected or score-coded.

In some embodiments, the method includes, responsive to the capturedchord sounding gestures, visually displaying strings of the multi-stringmusical instrument on the multi-touch sensitive display incorrespondence with the user's performance and at positions beneath oneor more of the respective finger contacts. In some cases, selection andpositioning of the visually displayed strings are dynamically varied incorrespondence with the captured chord sounding gestures. In some cases,fret positions and lateral extent of the visually displayed strings aredynamically varied in correspondence with the captured chord soundinggestures.

In some embodiments, the method further includes presenting the user ofthe synthetic musical instrument with visual cues on the multi-touchsensitive display, the presented visual cues indicative of temporallysequenced chord selections to be sounded by the user in accord with amusical score. In some cases, the chord-indicative visual cues arepresented using a symbology evocative of shape and orientation of fingercontacts that define a particular chord sounding gesture cued. In somecases, the presented visual cues include at least some visual cuesindicative of individual note selections. In some cases, notwithstandingmulti-string nature of the synthetic musical instrument, thepresentation of the visual cues is piano roll style, wherein individualvisual cues travel toward pitch indicative positions along a laterallyextended sounding zone of the multi-touch sensitive display. In somecases, lateral extent of the sounding zone is dynamically allocated sothat pitch range of available notes and spacing between note selectivepositions distributes a current contextually relevant set of soundingpositions along the available lateral extent of a multi-touch sensitivedisplay.

In some embodiments, the method further includes determiningcorrespondence of respective captured chord sounding gestures with thevisually cued chord selections; and grading the user's performance atleast in part based on the determined correspondences, wherein thedetermined correspondences include (i) a measure of temporalcorrespondence of a particular chord sounding gesture with arrival of avisual cue in the sounding zone and (ii) a measure correspondence of thechord sounded with the visual cue.

In some cases, the stream of chord indicative inputs includesconstituent pluralities of note selection inputs consistent with atemporal sequence of the captured and distinguished chord soundinggestures. In some cases, the digital synthesis includes a sample-basedsynthesis of constituent notes of the captured chord. In some cases, thedigital synthesis includes a sample-based synthesis of the capturedchord. In some cases, the audible rendering includes modeling acousticresponse for the multi-string instrument; and driving the modeledacoustic response with inputs corresponding to the captured chordsounding gestures.

In some embodiments, the synthetic multi-string instrument is a guitar,and the visual cues travel across the multi-touch sensitive display andrepresent, in one dimension of the multi-touch sensitive display,desired finger contacts along a fretless single-string analog of themulti-string instrument in accordance with notes of the score and, in asecond dimension generally orthogonal to the first, temporal sequencingof the desired finger contacts paced in accord with a current tempo.

In some embodiments, the method further includes presenting on themulti-touch sensitive display a lesson plan of exercises, wherein thecaptured chord selection gestures correspond to performance by the userof a particular one of the exercises; and advancing the user to a nextexercise of the lesson plan based on a grading of the user's performanceof the particular exercise.

In some embodiments, the portable computing device includes acommunications interface, and the method further includes transmittingan encoded stream of the note selection gestures via the communicationsinterface for rendering of the performance on a remote device.

In some embodiments, the method further includes geocoding thetransmitted gesture stream; and displaying a geographic origin for, andin correspondence with audible rendering of, another user's performanceencoded as another stream of notes sounding gestures received via thecommunications interface directly or indirectly from a remote device.

In some embodiments, the portable computing device is selected from thegroup of: a compute pad, a game controller, a personal digital assistantor book reader, and a mobile phone or media player.

In some embodiments in accordance with the present invention(s), acomputer program product is encoded in one or more media and includesinstructions executable on a processor of the portable computing deviceto cause the portable computing device to perform the method. In somecases, the media are readable by the portable computing device orreadable incident to a computer program product conveying transmissionto the portable computing device.

In some embodiments in accordance with the present invention(s), anapparatus includes a portable computing device having a multi-touchsensitive display and machine readable code executable on the portablecomputing device to implement a synthetic musical instrument. Themachine readable code includes instructions executable to capture chordsounding gestures indicated by a user on the multi-touch sensitivedisplay, wherein the capture of chord sounding gestures includes, forparticular and respective chords so gestured, registration of contact bymultiple fingers of the user's hand and distinguishing betweenrespective geometries of the multiple-finger contacts, wherein thecaptured chord sounding gestures are characterized by the respectivemultiple-finger contact geometries rather than solely by positionalregistrations of individual finger contacts relative to string or fretpositions. The machine readable code is still further executable toaudibly render the user's performance on the portable computing deviceusing, as an input to a digital synthesis of the synthetic musicalinstrument executing on the portable computing device, a gesture streamthat includes the captured chord sounding gestures, wherein the gesturestream, and not the musical score itself, drives the digital synthesis.In some embodiments the apparatus is embodied as one or more of acompute pad, a game controller, a handheld mobile device, a mobilephone, a personal digital assistant, a media player and a book reader.

In some embodiments in accordance with the present invention(s), acomputer program product is encoded in media and includes instructionsexecutable to implement a synthetic multi-string musical instrument on aportable computing device having a multi-touch display interface. Thecomputer program product encodes and includes instructions executable bythe portable computing device to capture chord sounding gesturesindicated by a user on the multi-touch sensitive display, wherein thecaptured chord sounding gestures include, for particular and respectivechords so gestured, contact by multiple fingers of the user's hand andwherein the captured chord sounding gestures are characterized byrespective geometries of the multiple-finger contacts rather than solelyby positional registrations of individual finger contacts relative tostring or fret positions. The instructions are executable by theportable computing device to distinguish between respective of themultiple-finger contact geometries gestured by the user and basedthereon to pass a stream of chord indicative inputs to a digitalsynthesis of the multi-string musical instrument executing on theportable computing device. In addition, the instructions are executableby the portable computing device as the digital synthesis to audiblyrender a performance in correspondence with the stream of captured chordindicative gestures.

In some embodiments, the instructions are executable by the portablecomputing device to distinguish between respective of themultiple-finger contact geometries, include instruction sequences todistinguish two-, three- and four-touch gestures. In some embodiments,the instructions are executable by the portable computing device tocapture chord sounding gestures capture both multi-touch chord selectiongestures and a traveling-touch, strum-type gesture.

In some embodiments, the computer program product further includesinstructions executable to determine correspondence of respectivecaptured chord sounding gestures with the chord selections visually cuedin the sounding zone and to grade the user's performance based on thedetermined correspondences. In some case, the media are readable by theportable computing device or readable incident to a computer programproduct conveying transmission to the portable computing device.

Score Coded Performance Effect Cues

In some embodiments in accordance with the present invention(s), amethod includes using a portable computing device as a multi-stringsynthetic musical instrument and presenting a user of the syntheticmusical instrument with visual cues on a multi-touch sensitive displayof the portable computing device. The presented visual cues areindicative of temporally sequenced note selections to be sounded by theuser in accord with a musical score, wherein notwithstandingmulti-string nature of the synthetic musical instrument, thepresentation of the visual cues is piano roll style, wherein individualvisual cues travel toward pitch indicative positions along a laterallyextended sounding zone of the multi-touch sensitive display, and whereinat least one of the presented visual cues indicates a string bendperformance effect to be sounded by the user in accord with the musicalscore. The method further includes capturing note sounding gesturesindicated by the user using the multi-touch sensitive display andaudibly rendering the performance on the portable computing device usingthe captured gesture stream as an input to a digital synthesis of thesynthetic musical instrument executing on the portable computing device,wherein the captured gesture stream, and not the musical score itself,drives the digital synthesis. In some cases, the captured note soundinggestures include finger contacts at pitch selective positions along thelaterally extended sounding zone.

In some embodiments, the captured note sounding gestures for the stringbend performance effect include finger contact and in-contact travel onthe multi-touch sensitive display in a direction generally orthogonal tolateral extent of the sounding zone. In some cases, the audiblerendering in correspondence with a captured string bend indicativegesture varies pitch from an initial pitch to a pitch corresponding to anote selection coded in the musical score. In some cases, the audiblerendering in correspondence with a captured string bend indicativegesture varies pitch from a pitch corresponding to a note selectioncoded in the musical score.

In some embodiments, lateral extent of the note sounding zone isdynamically allocated so that pitch range of available notes and spacingbetween note selective positions distributes a current contextuallyrelevant set of note sounding positions along the available lateralextent of a multi-touch sensitive display. In some embodiments, thecaptured note sounding gestures include, for a particular chord visuallycued in accordance with the musical score, contact by multiple fingersof the user's hand.

In some embodiments, the method further includes distinguishing betweenplural multi-finger contact geometries as chord indicative gestures and,in correspondence with a captured chord indicative gesture, audiblyrendering a corresponding multi-string chord using the digitalsynthesis.

In some cases, at least one of the presented visual cues indicates avibrato performance effect in accord with the musical score; and themethod further includes capturing and including a vibrato indicativegesture in the gesture stream supplied as input to the digital synthesisof the synthetic string instrument executing on the portable computingdevice. In some cases, capture of the vibrato indicative gesture uses anon-board accelerometer of the portable computing device.

In some embodiments, the method further includes determiningcorrespondence of respective captured note sounding gestures with thenote selections visually cued in the sounding zone and grading theuser's performance based on the determined correspondences. In somecases, the determined correspondences include: a measure of temporalcorrespondence of a particular note sounding gesture with arrival of avisual cue in the sounding zone; and a measure of note selectioncorrespondence of the particular note sounding gesture with the visualcue. In some cases, the determined correspondences include a measure oftemporal correspondence of the string bend performance effect witharrival of the corresponding visual cue in the sounding zone.

In some embodiments, the synthetic multi-string instrument is a guitar,and the visual cues travel across the multi-touch sensitive display andrepresent, in one dimension of the multi-touch sensitive display,desired finger contacts along a fretless single-string analog of themulti-string instrument in accordance with notes of the score and, in asecond dimension generally orthogonal to the first, temporal sequencingof the desired finger contacts paced in accord with a current tempo. Insome cases, the sounding zone corresponds generally to a generallylinear display feature on the multi-touch sensitive display toward oracross which the visual cues travel. In some cases, the captured notesounding gestures are indicative of both string excitation and pitchselection for the excited string.

In some embodiments, the method further includes presenting on themulti-touch sensitive display a lesson plan of exercises, wherein thecaptured note selection gestures correspond to performance by the userof a particular one of the exercises and advancing the user to a nextexercise of the lesson plan based on a grading of the user's performanceof the particular exercise.

In some embodiments, the portable computing device includes acommunications interface, and the method further includes transmittingan encoded stream of the note selection gestures via the communicationsinterface for rendering of the performance on a remote device.

In some embodiments, the method further includes geocoding thetransmitted gesture stream and displaying a geographic origin for, andin correspondence with audible rendering of, another user's performanceencoded as another stream of notes sounding gestures received via thecommunications interface directly or indirectly from a remote device.

In some cases, the audible rendering includes modeling acoustic responsefor the multi-string instrument and driving the modeled acousticresponse with inputs corresponding to the captured note soundinggestures. In some cases, the portable computing device is selected fromthe group of: a compute pad, a game controller, a personal digitalassistant or book reader, and a mobile phone or media player.

In some embodiments, a computer program product is encoded in one ormore media and includes instructions executable on a processor of theportable computing device to cause the portable computing device toperform one or more of the aforementioned methods. In some cases, mediumor media is (are) readable by the portable computing device or readableincident to a computer program product conveying transmission to theportable computing device.

In some embodiments in accordance with the present invention, anapparatus includes a portable computing device having a multi-touchdisplay interface and machine readable code executable on the portablecomputing device to implement a synthetic musical instrument. Themachine readable code includes instructions executable to present a userof the synthetic musical instrument with visual cues on the multi-touchsensitive display. The presented visual cues are indicative oftemporally sequenced note selections to be sounded by the user in accordwith a musical score, wherein notwithstanding multi-string nature of thesynthetic musical instrument, the presentation of the visual cues ispiano roll style, wherein individual ones of the visual cues traveltoward pitch indicative positions along a laterally extended soundingzone of the multi-touch sensitive display. At least one of the presentedvisual cues indicates a string bend performance effect to be sounded bythe user in accord with the musical score. The machine readable code isfurther executable to capture note sounding gestures indicated by theuser using the multi-touch sensitive display and to audibly render theperformance on the portable computing device using the captured gesturestream as an input to a digital synthesis of the synthetic musicalinstrument executing on the portable computing device, wherein thecaptured gesture stream, and not the musical score itself, drives thedigital synthesis.

In some embodiments, the machine readable code is further executable tovary (in correspondence with a captured string bend indicative gesture)an audibly rendered pitch to or from a pitch corresponding to a noteselection coded in the musical score. In some embodiments, the machinereadable code is further executable to distinguish between pluralmulti-finger contact geometries as chord indicative gestures and toaudibly render (in correspondence with a captured chord indicativegesture) a corresponding multi-string chord using the digital synthesis.In some cases, the apparatus is embodied as one or more of a computepad, a handheld mobile device, a mobile phone, a personal digitalassistant, a smart phone, a media player and a book reader.

In some embodiments in accordance with the present invention, a computerprogram product is encoded in media and includes instructions executableto implement a synthetic multi-string musical instrument on a portablecomputing device having a multi-touch display interface. The computerprogram product encodes and includes instructions executable by theportable computing device to present a user of the synthetic musicalinstrument with visual cues on the multi-touch sensitive display, thepresented visual cues indicative of temporally sequenced note selectionsto be sounded by the user in accord with a musical score, whereinnotwithstanding multi-string nature of the synthetic musical instrument,the presentation of the visual cues is piano roll style, whereinindividual ones of the visual cues travel toward pitch indicativepositions along a laterally extended sounding zone of the multi-touchsensitive display, and wherein at least one of the presented visual cuesindicates a string bend performance effect to be sounded by the user inaccord with the musical score. The computer program product furtherencodes and includes instructions executable by the portable computingdevice to capture note sounding gestures indicated by the user using themulti-touch sensitive display and instructions executable by theportable computing device to audibly render the performance on theportable computing device using the captured gesture stream as an inputto a digital synthesis of the synthetic musical instrument executing onthe portable computing device, wherein the captured gesture stream, andnot the musical score itself, drives the digital synthesis.

In some embodiments, the computer program product further includes andencodes instructions executable to determine correspondence ofrespective captured note sounding gestures with the note selectionsvisually cued in the sounding zone and to grade the user's performancebased on the determined correspondences. In some cases, the medium ormedia is (are) readable by the portable computing device or readableincident to a computer program product conveying transmission to theportable computing device.

These and other embodiments in accordance with the present invention(s)will be understood with reference to the description herein as well asthe drawings and appended claims which follow.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example and notlimitation with reference to the accompanying figures, in which likereferences generally indicate similar elements or features.

FIGS. 1 and 2 depict performance uses of a portable computing devicehosted implementation of a synthetic guitar in accordance with someembodiments of the present invention. FIG. 1 depicts an individualperformance use and FIG. 2 depicts note sequences (including a stringbend effect) visually cued in accordance with a musical score.

FIGS. 3A, 3B and 3C illustrate spatio-temporal cuing aspects of a userinterface design for a synthetic guitar instrument in accordance withsome embodiments of the present invention. Note that frame of referenceis rotated by 180 degrees from that illustrated in FIGS. 1 and 2 toinstead present a view typical of that presented to the user musician.Accordingly, in the frame of reference illustrated in FIGS. 3A, 3B and3C, note cues travel upward on the illustrated page and string bendinggestures cue finger contact that travels downward on the illustratedpage.

FIG. 4 is a functional block diagram that illustrates capture andencoding of user gestures corresponding to a sequence of note and/orchord soundings in a performance on a synthetic guitar-type musicalinstrument, together with acoustic rendering of the performance inaccordance with some embodiments of the present invention.

FIGS. 5 and 6 depict performance uses of a portable computing devicehosted implementation of a synthetic guitar in accordance with someembodiments of the present invention in which chord gesturing issupported. FIG. 5 depicts an individual performance use and FIG. 6illustrates and annotates a chord selection gesture (a C major chord)and a strum gesture on a multi-touch sensitive display of the portablecomputing device.

FIGS. 7A, 7B and 7C illustrate and annotate several variations on achord selection gesture (here a G major chord) to emphasize that, inaccordance with some embodiment of the present invention(s), themulti-touch selection gesture for a given chord may be characterized bygeneral shape and general orientation, rather than with reference to anyprecise positional registration with string or fret positions on screen.FIG. 7D illustrates and annotates one presentation of the G major chordselection gesture on a multi-touch sensitive display of a portablecomputing device. FIG. 7E illustrates and annotates one presentation ofthe D major chord selection gesture on a multi-touch sensitive displayof the portable computing device.

FIG. 8 depicts a decision tree for distinguishing multi-touch chordselection gestures based on number of touches, general shape andorientation and in some cases, horizontal position. Variants for rootand successive inversions of gestured chord selections are alsodepicted.

FIG. 9 is a functional block diagram that further illustrates, inaddition to gesture capture and performance grading (previouslydescribed), optional communication of performance encodings and/orgrades as part of a game play or competition framework, social networkor content sharing facility in accordance with some embodiments of thepresent invention.

FIG. 10 is a functional block diagram that illustrates capture, encodingand transmission of a gesture stream (or other) encoding correspondingto a user performance on a synthetic multi-string instrument togetherwith receipt of such encoding and acoustic rendering of the performanceon a remote device.

FIG. 11 is a network diagram that illustrates cooperation of exemplarydevices in accordance with some embodiments of the present invention.

Skilled artisans will appreciate that elements or features in thefigures are illustrated for simplicity and clarity and have notnecessarily been drawn to scale. For example, the dimensions orprominence of some of the illustrated elements or features may beexaggerated relative to other elements or features in an effort to helpto improve understanding of embodiments of the present invention.

DETAILED DESCRIPTION

In the description that follows, we detail certain exemplaryimplementations of synthetic multi-string musical instruments. In doingso, we use a familiar 6-string guitar as a teaching example forimplementations in accordance with some embodiments of the presentinvention(s). Specifically, and withstanding the multi-string nature ofcertain instruments that may be synthesized, alternative visual cuingand gesture expression models have been found to be more suitable forthe limited touch-screen real-estate provided by mobile handheld devicessuch as phones, media players, gaming controllers, etc., at least insome cases or modes of operation. Accordingly, a “piano roll” style setof visual cues has been adopted to provide a song-mode user of asynthetic guitar-type (multi-string) musical instrument with note and/orchord selection cues (in accordance with a musical score). Thus, thevisual cuing and note expression gesture model for a multi-stringfretted instrument can rationalized and simplified to a laterallyextended note sounding zone, without regard to fretted quantization ofpitches or allocations to respective strings.

The synthetic multi-string musical instrument captures a stream ofexpressive gestures indicated on a multi-touch sensitive display fornote/chord soundings and associated performance effects andembellishments. While the visual cues are driven by a musical score andrevealed/advanced at a current performance tempo, it is the user'sgestures that actually drive the audible performance rendering.Opportunities for user expression (or variance from score) include onsetand duration of note soundings, tempo changes, as well as uncued stringbend effects, vibrato, etc.

By using a fretless, laterally extended note sounding zone, lateralextent can be dynamically allocated so that the pitch range of availablenotes (and spacing between note selective positions) can be allocated tolaterally distribute a current contextually relevant set of notesounding positions along the available lateral extent of a multi-touchsensitive display. In general, range and/or spacing may be dynamicallyselected based a note set employed in a current song selection orportion thereof. In this way, visual cues and user note soundinggestures may be distributed across the multi-touch sensitive display ina manner that makes it easier for the user musician to play. Thesynthetic guitar may allocate finger space to each note, but need onlyallocate space for the note set that is necessary to play the currentselection or song (or to play at a current point within a currentselection or song).

Particularly in freestyle modes of play, it is possible to provide userswith a gesturing mechanism to sound chords without having to registerprecisely accurate multi-touch screen contacts. This is especiallyimportant for mobile phone, media player and game controllerembodiments, where there is generally limited real-estate to display six(6) or more strings, and user fingers are generally too fat to preciselycontact such strings. Instead, for a given key and context, most of theimportant chords have a distinguishing “shape” and location in terms ofthe fingers that contact the strings and strings that are contacted.These and other aspects will be understood based on specificimplementations, embodiments, and teaching examples that follow.Notwithstanding, reliance on a conventional 6-string guitar as anexemplary physical world analogue for certain concrete realizations ofsynthetic multi-string musical instruments described herein, it will beunderstood that inventions described and claimed are not limited to anyparticular musical instrument, let alone to implementations that providea synthetic 6-string guitar.

FIGS. 1 and 2 depict performance uses of a portable computing devicehosted implementation of a synthetic guitar in accordance with someembodiments of the present invention. FIG. 1 depicts an individualperformance use and FIG. 2 depicts note sequences (including a stringbend effect) visually cued in accordance with a musical score. In theillustrated frame of reference, visual cues descend toward a soundingzone, where user fingerings (e.g., touch screen contacts) are capturedas indicative gestures to trigger the sounding or voicing notes that(depending on facility of the user musician) generally correspond tovisual cues and underlying musical score.

FIGS. 3A, 3B and 3C illustrate spatio-temporal cuing aspects of a userinterface design for a synthetic guitar instrument in accordance withsome embodiments of the present invention. Note that frame of referenceis rotated by 180 degrees from that illustrated in FIGS. 1 and 2 toinstead present a view typical of that presented to the user musician.Accordingly, in the frame of reference illustrated in FIGS. 3A, 3B and3C, note cues travel upward on the illustrated page and string bendinggestures cue finger contact that travels downward on the illustratedpage.

More specifically, FIG. 3A illustrates temporally sequenced note cuespresented, at a current tempo, in correspondence with an underlyingmusical score. In general, higher pitched notes soundable by thesynthetic guitar present as visual cues toward the right side of theillustrated screen shots, while lower pitched notes present as visualcues toward the left side. One of the note cues (note cue 311) is in thesounding zone 301, suggesting to the user musician that (based on thecurrent target tempo) the corresponding note is to be sounded by fingercontact indicative of string excitation. Timing, attack and duration ofthe actual finger contact gesture drive digital synthesis and,accordingly, the audible rendering. Lateral position of the actualfinger contact can affect the sounded pitch, although in some cases,embodiments or modes, lateral variance from the visually cued notesounding position may be tolerated or ignored altogether. Successivenote cues 312, 313, 314 and 315 (in correspondence with the musicalscore) are depicted along their travel paths toward the sounding zone.One of those successive note cues (the arrow-type cue) cues a stringbend to (or from) a score coded note selection.

FIG. 3B illustrates a note cue 321 that includes a string bendperformance effect indication suggestive to the user (as visual cuing)of a further string (or pitch) bend performance effect to be sounded bythe user musician relative to the visually cued note. To sound thestring bend performance effect, the user musician, after contacting themulti-touch sensitive display, transits his/her finger contact in adirection generally orthogonal to the sounding zone to gesture a stringbend. In correspondence with the string bend gesture, digital synthesiseffectively varies tension of the modeled vibrating string or, in thecase of a wave table synthesis, resampling, interpolation andcross-fading may be employed to effectuate a pitch shift in the audiblerendering. A visual indication 322 on screen provides additionalfeedback to the user musician.

Finally, FIG. 3C illustrates temporal and pitch indicative axes of thevisual cuing model and lateral extent of the note sounding zone. Lateralextent of the note sounding zone is dynamically allocable to a currentpitch range of available notes and spacing between note selectivepositions. More specifically, the synthetic guitar distributes a currentcontextually relevant set of note sounding positions along the availablelateral extent of the multi-touch sensitive display. In this way,limited display and finger contact real estate may be allocated tovisual cuing and gesture capture.

Although illustrations of FIGS. 3A, 3B and 3C focus on individual notecues and sounding gestures, it will be understood that (in some cases,embodiments or modes) captured note sounding gestures may include, for aparticular chord visually cued in accordance with the musical score,contact by multiple fingers of the user's hand. Chord sounding gesturecapture is contemplated and described in greater detail elsewhereherein, but adaptations of the above-described techniques to score-codedand visually cued chord soundings will be understood by persons ofordinary skill in the art having access to the present disclosure.

FIG. 4 is a functional block diagram that illustrates capture of usergestures corresponding to a sequence of note and/or chord sounding cuesin a performance on a synthetic guitar-type musical instrument (e.g.,Magic Guitar Application 450 executing on portable computing device401), together with acoustic rendering of the performance in accordancewith some embodiments of the present invention. Note or chord soundingsgestured by the user musician at touch screen/display 414 of portablecomputing device 401 are typically (at least in some modes of operationand for some user musicians) in correspondence with visually presentednote cues on touch screen/display 414 and are, in turn, captured (453)and used to drive a digital synthesis (454) of acoustic response of thesynthetic guitar. Such visual cues (recall FIGS. 3A, 3B, 3C) aresupplied in accordance with a musical score (notes, chords, meter andperformance effects and embellishments) stored at least transiently instorage 456 and at a rate that is based on a current tempo. In someembodiments that rate may be continuously adapted (using a tempo control459) based on techniques described relative to a synthetic piano-typemulti-string musical instrument in commonly-owned, co-pending U.S.patent application Ser. No. 13/664,939, filed Oct. 31, 2012, entitled“SYNTHETIC MUSICAL INSTRUMENT WITH PERFORMANCE-AND/OR SKILL-ADAPTIVESCORE TEMPO” and naming Hamilton, Chaudhary, Lazier and Smith asinventors. Application Ser. No. 13/664,939 is incorporated by referenceherein.

In other embodiments or modes, tempo of note or chord cues may remaingenerally fixed in accord with a musical score, though typicallyallowing (in some cases or embodiments) for a stall in cases where theuser musician fails to effectuate a responsive note or chord soundinggesture on touch screen/display 414. As will be also be appreciatedbased on the present description, in some embodiments or modes ofoperation, note and/or chords may be sounded in freestyle modes orotherwise without regard to a score-coded sequence. In each case, noteor chord sounding gestures (rather than elements of the score itself)drive the digital synthesis (e.g., here of a 6-string guitar). Forpurposes of understanding suitable implementations, any of a wide rangeof digital synthesis techniques may be employed to drive audiblerendering (411) of the user musician's performance via a speaker orother acoustic transducer (412) or interface thereto.

In general, the audible rendering can include synthesis of tones,overtones, harmonics, perturbations and amplitudes and other performancecharacteristics based on the captured gesture stream. Note that as usedherein the captured gesture stream will be broadly understood and mayencompass a multiplicity of gestures and gesture types such as, forexample, single string note soundings such as might be expressed (on aphysical instrument analogue) by the combined actions of a finger downposition on a fret board and a pluck of the shortened string. In somecases or embodiments, particularly relative to user interface designs ofthe type illustrated with respect to a single string, piano roll-styleuser interface such as illustrated above relative to FIGS. 3A, 3B and3C, the user gesture analogues of selection and sounding may beconflated in the gesture notation and capture. In such cases, anexpressed and captured touch gesture may sound and select a particularnote.

Likewise, in certain chord sounding gesture implementations described inthe sections that follow, chord selection and sounding may be conflated.Nonetheless, it is to be understood that, more generally and in somecases, modes or embodiments, selection and sounding of a note or chordmay be expressed as separate (though typically coordinated) gestures.For example, by a collection of finger contacts of one hand onmulti-touch sensitive display 414 a user musician may gesture a chordselection and with a further travelling-touch gesture using a finger ofthe other hand may gesture a “strum” that together with the selectionwill be understood to constitute a chord sounding gesture or gestures.Note that, in some cases, embodiments or modes, an automatic strum maybe provided for the user musician and introduced into the gesture streambased on score-coded meter. Furthermore, some embodiments may supportornamentation gestures such as the previously described string bendgesture or, for example, a vibrato gesture expressed by the usermusician as shaking detected by an accelerometer or other sensor 417 ofportable computing device 401.

In each case, the gesture stream drives the digital synthesis 454 whichin turn may be rendered. In some cases, an audible rendering is on thevery device on which the musical performance is captured. In somesituations or embodiments, the gesture stream encoding is conveyed to aremote device whereupon an audible rendering converts a synthesizedsignal to acoustic energy. In some cases, an appropriate CODEC isemployed to suitably encode the resulting audio rendering for storageand/or transmission.

The digital synthesis (454) of a 6-string guitar (or multi-stringmusical instrument) allows the user musician to control an actualexpressive model (whether by wavetable synthesis, physical acousticmodeling or otherwise) using multi-sensor interactions as inputs. Notethat digital synthesis (554) is, at least for full synthesis modes,driven by the user musician's note sounding gestures, rather than bymere tap triggered release of the next score coded note. In this way,the user is actually causing the sound and controlling the timing,decay, pitch, quality and other characteristics of notes (includingchords) sounded. A variety of computational techniques may be employedand will be appreciated by persons of ordinary skill in the art. Forexample, exemplary techniques include wavetable or FM synthesis.

Wavetable or FM synthesis is generally a computationally efficient andattractive digital synthesis implementation for piano-type musicalinstruments such as those described and used herein as primary teachingexamples. However, and particularly for adaptations of the presenttechniques to syntheses of certain types of multi-string instruments(e.g., unfretted multi-string instruments such as violins, violas cellosand double bass), physical modeling may provide a livelier, moreexpressive synthesis that is responsive (in ways similar to physicalanalogs) to the continuous and expressively variable excitation ofconstituent strings. For a discussion of digital synthesis techniquesthat may be suitable in other synthetic instruments, see generally,commonly-owned co-pending application Ser. No. 13/292,773, filed Nov.11, 2011, entitled “SYSTEM AND METHOD FOR CAPTURE AND RENDERING OFPERFORMANCE ON SYNTHETIC STRING INSTRUMENT” and naming Wang, Yang, Ohand Lieber as inventors, which is incorporated by reference herein.

Chord Sounding Gesture Capture

In some embodiments and particularly in freestyle modes of play, it ispossible to provide users with a gesturing mechanism to sound chordswithout having to register precisely accurate multi-touch screencontacts. This is especially important for mobile phone, media playerand game controller embodiments, where there is generally limitedreal-estate to display six (6) or more strings, and user fingers aregenerally too fat to precisely contact such strings. Instead, for agiven key and context, most of the important chords will be understoodto have a distinguishing “shape,” orientation and coarse-grain positionin terms of the fingers that contact the strings and strings that arecontacted. Based on such shapes, orientations and course positionings, achord gesturing framework and capture techniques have been developedthat has been found to be reasonably intuitive for user musicians andamateurs alike, but also largely independent of an precise positional orscaling requirements for touches relative to string, fret positions oreach other.

The basic concept is that for a given key and context, most of theimportant chords have a distinguishable “shape” and location in terms ofthe fingers that contact the strings and which strings are contacted. Byrecognizing in multi-touch gesture capture, that an E major chord isgestured as an obtuse triangle of finger contacts at the top of thestrings (screen), an A minor chord has the same shape but is lower onthe screen, an A major chord is gestured as a linear arrangement offinger contacts near the bottom of the screen, a C major chord isgestured as an upward diagonal arrangement of finger contacts at thecenter of the screen, and a D major chord is gestured as a nearlyequilateral triangle of finger contacts at the bottom of the screen,etc., it is possible to reliably guess at the most likely chord soundingexpressed by the user. The user doesn't have to hit strings and fretsexactly. Indeed, positional registration along respective string displayartifacts need not matter at all, or can a little but not as criticallyas real frets. Rather, the user may simply gesture finger contacts inroughly the right shape at roughly the right portion of the screen.Gesture recognition and capture algorithm(s) identify the sounded chordand supply appropriate gesture-driven inputs to a multi-stringsynthesis. In some embodiments, strings are displayed or dynamicallyrevealed beneath the finger contacts in correspondence with therecognized chord to visually reinforce the chord indication.

In FIGS. 5 and 6, we depict performance uses of a portable computingdevice (e.g., portable computing device 401, recall FIG. 4) hosting andexecuting a synthetic software implementation of a six (6) string guitar(e.g., as Magic Guitar Application 450) in accordance with someembodiments of the present invention in which chord gesturing issupported. Specifically, FIG. 5 depicts an individual performance use inwhich a three-touch chord indicative gesture has been distinguished bythe synthetic multi-string instrument as a C major chord andcorresponding constituent strings of the chord have been presented (ondisplay) generally beneath the actual touch points. Accordingly, spacingof the displayed strings is scaled (in some cases non-uniformly) andpositioning of collection of strings in the vertical dimension (i.e.,vertical from a user-musician perspective consistent with the view ofFIG. 5) correspond to the actual user gesturing. A travelling touchstrum gesture expressed using the index finger of the user-musician'sright hand as (in conjunction with the captured chord selection) beenprocessed as a sounding of the C major chord. Of course, as explainedelsewhere herein, the chord sounding may be gestured as a two-partgesture (selection and sounding as just described) or alternatively thestrum may be synthetic (e.g., automatically generated accord withscore-coded meter to sound whatever notes and/or chords are currentlyselected) or simply implicit in (and therefore conflated with) themulti-touch selection of the particular chord.

FIG. 6 illustrates and annotates a simplified screen image correspondingto the chord selection gesture 610 (for a C major chord) as justexplained together with a strum gesture 620 on a multi-touch sensitivedisplay of the portable computing device. A generally linear three-toucharrangement of finger contacts 611, 612 and 613 is distinguished as theC major chord illustrated, notwithstanding what would be somewhatimprecise correspondence of touch points to string and fret positionswere they represented by the synthetic multi-string instrumentconventionally as fixed positional registrations. Note that strings 640are positioned and revealed beneath actual touch points in response thechord sounding gesture. In the illustration of FIG. 6, in addition tothose strings selected in correspondence with the touch points (strings641) remaining strings including open strings 642 are revealed.Optionally, and in some situations, modes or embodiments, only theselected strings 641 may be revealed. Note that the strings revealed(including the remaining unselected strings) are adaptively positionedand spaced in correspondence with the actual string selection touchpoints. As a result, strings 640 need not be equally spaced. Consistentwith a physical 6-string guitar analogue, open string tunings (which canbe set based on tuning of the instrument) sound at pitches that are usedin the chord. Alternatively, tunings for the open strings 642 can bedynamically adapted (in the synthetic multi-string instrument) to thesounded chord. Based on quality of the chord sounding gesture expression(e.g., timing of onset or strumming relative to a score, sustenance ofthe selection gesture, expressed vibrato or other ornamentation), pointsmay be optionally awarded in skill demonstration or game play mode.

FIGS. 7A, 7B and 7C illustrate and annotate several variations (710,710B and 710C) in the touch point expression of a chord selectiongesture (here a G major chord) to emphasize that, in accordance withsome embodiments of the present invention(s), the multi-touch selectiongesture for a given chord may be characterized by general shape andgeneral orientation, rather than with reference to any precisepositional registration with string or fret positions on screen.Specifically, FIG. 7A illustrates a near ideal abstraction of the fingerpositioning for a G major chord as an upward (along strings) pointingtriangle of touch points 711, 712 and 713. FIG. 7B illustrates asomewhat tighter spacing of touch points with a slight counterclockwiserotation from the near ideal. Notwithstanding the differing coordinatesof the 710B variant touch points 711, 712 and 713, the syntheticmulti-string instrument recognizes the general shape and orientationfeatures that distinguish the G major chord selection/sounding gesture.Likewise, FIG. 7C illustrates still another spacing of touch points,this time more closely approximating an arrangement that might begestured by an experienced guitar player given its better correspondencewith string and fret spacings that would appear in a physical instrumentanalogue. Notwithstanding the differing coordinates of the 710C varianttouch points 711, 712 and 713 from the idealized abstraction, thesynthetic multi-string instrument again recognizes the general shape andorientation features that distinguish the G major chordselection/sounding gesture.

FIG. 7D illustrates and annotates one presentation (an idealizedabstraction 710 of the G major chord selection/sounding gesture) on amulti-touch sensitive display of a portable computing device. As before,generally upward-pointing (in string direction) triangular three-toucharrangement of finger contacts 711, 712 and 713 is distinguished as theG major chord illustrated, notwithstanding what would be somewhatimprecise correspondence of touch points to string and fret positionswere they represented by the synthetic multi-string instrumentconventionally as fixed positional registrations. As before, strings(here strings 740) are positioned and revealed beneath actual touchpoints in response the chord sounding gesture. In addition to thosestrings selected in correspondence with the touch points remainingstrings including open strings are revealed and, consistent with anon-uniform string spacing rationalizable with the particular expressionof the G major chord selection/sounding gesture abstraction, three openstrings are packed between touch points 711 and 713. Again optionally,and in some situations, modes or embodiments, only the selected stringsmay be revealed. As before, consistent with a physical 6-string guitaranalogue, open string tunings (whose tuning can be set based on tuningof the instrument) sound at pitches that are used in the chord.Alternatively, tunings for the open strings can be dynamically adapted(in the synthetic multi-string instrument) to the sounded chord. Again,based on quality of the chord sounding gesture expression, points may beoptionally awarded in skill demonstration or game play mode.

Finally, FIG. 7E illustrates and annotates an abstraction 730 of a Dmajor chord selection/sounding gesture) on a multi-touch sensitivedisplay of a portable computing device. In this case, generallydownward-pointing (in string direction) triangular three-toucharrangement of finger contacts 731, 732 and 733 is distinguished as theD major chord illustrated, again notwithstanding what would be somewhatimprecise correspondence of touch points to string and fret positionswere they represented by the synthetic multi-string instrumentconventionally as fixed positional registrations. As before, strings arepositioned and revealed beneath actual touch points in response thechord sounding gesture. In addition to those strings selected incorrespondence with the touch points remaining strings including openstrings are revealed although, as before and optionally in somesituations, modes or embodiments, only the selected strings may berevealed. Again, based on quality of the chord sounding gestureexpression, points may be optionally awarded in skill demonstration orgame play mode.

Chord Selection Decision Tree and Exemplary Code

FIG. 8 depicts a decision tree for distinguishing multi-touch chordselection/sounding gestures based on number of touches, general shapeand orientation and in some cases, horizontal position. Variants forroot and successive inversions of gestured chord selections are alsodepicted. In addition to the C major, G major and D major chordselection/sounding gestures explained in detail above relative toscreenshots, exemplary expressions of additional chords (including minorchords, major and dominant seventh (7^(th)) variations and inversionsdistinguishable by an exemplary embodiment of a synthetic multi-stringmusical instrument (e.g., Magic Guitar Application 450 executing onportable computing device 401, recall FIG. 4) are depicted.

To facilitate understanding of exemplary implementations the techniquesdescribed herein, the following pseudo-code snippets are provided andare illustrative of an embodiment of gesture capture (see e.g., gesturecapture block 453, recall FIG. 4) in which consistent with the decisiontree of FIG. 8, multi-touch chord selection gestures are distinguishedbased on number of touches, general shape and orientation and in somecases, horizontal position. Variants for root and successive inversionsof gestured chord selections are also distinguished. The illustratedpseudo-code assumes an array-type data structure allTouches to which amulti-touch API and related services adds elements of touches in theorder in which they send a touchesBegan notification. Individual touchesare removed from the array when they send a touchesEnded notificationand any remainder of the array is packed to the front.

Based on the foregoing, an exemplary two-touch chord sounding (orselection) gestures may be distinguished from one another based on thefollowing touch coordinate based calculations in which both orientationof a line traceable through the touch coordinates and horizontalplacement of the horizontal midpoint of the two-touch gesture.Specifically, E minor, A minor, E major and A major chords may bedistinguished follows:

if( numberOfTouches == 2 ) {  Point touchA = allTouches[0];  PointtouchB = allTouches[1];  // check if x similarity is closer than ysimilarity  if( abs(touchA.x − touchB.x) < abs(touchA.y − touchB.y) ) {  // the x points are closer, consider this vertical   // if the medianx is on the left side vs the right side,   // play a different chord  if ( median(touchA.x,touchB.x) < ScreenWidth ) {    setupChord(E_Minor )   }   else {    setupChord( A_Minor )   }  }  else {   // they points are closer, consider this diagonal/horizontal   // if themedian x is on the left side vs the right side,   // play a differentchord   if ( median(touchA.x,touchB.x) < ScreenWidth ) {    setupChord(E_Major )   }   else {    setupChord( A_Major )   }  } }

Likewise, touch coordinates for three-touch chord sounding (orselection) gestures are analyzed to identify features consistent withgeneral shapes and orientations which are, in turn, used to distinguishD major, G major, C major and F major chord indicative gestures from oneanother.

. . . else if( numberOfTouches == 3 ) {  // first order the touchesascending along the x dimension  // of the screen  xOrderedTouches =sortInXDimensionAscending( allTouches );  Point touchA =xOrderedTouches[0];  Point touchB = xOrderedTouches[1];  Point touchC =xOrderedTouches[2];  // now that the points are ordered we can makesimple  // comparisons for shapes  if ( touchB.y > touchA.y &&touchB.y > touchC.y ) {   // triangle pointed upwards   setupChord(G_Major )  }  else if ( touchB.y <= touchA.y && touchB.y <= touchC.y ) {  // triangle pointed downwards   setupChord( D_Major )  }  else if (touchA.y <= touchB.y && touchB.y <= touchC.y ) {   // line slopingupwards from left to right   setupChord( C_Major )  }  else if (touchA.y >= touchB.y && touchB.y >= touchC.y ) {   // line slopingdownwards from left to right   setupChord( F_Major )  } }

Finally, four-touch chord sounding gesture handling builds on an initialcharacterization of the main triad or major chord using techniques suchas described above relative to the three-touch chords. From there, thetouch coordinates for successive fourth-touch can be compared tocoordinates for the constituent touches of the already characterizedthree-touch main triad. Based on the position of the fourth touchrelative to the other constituents, major and dominant 7^(th) variants(or extensions) of the major chord may be distinguished as follows:

. . . else if( numberOfTouches == 4 ) {  // *** start off with identicalcode to numberOfTouches == 3 to set up the main triad using first threetouch points *** . . . see above if statement for numberOfTouches == 3 .. . // get the fourth touch, which we know to be the most recent //touch added Point touchD = xOrderedTouches[3];  if ( touchD.x > touchA.x&& touchD.x > touchB.x &&    touchD.x > touchC.x ) // fully to the right{  addExtensionToCurrentChord( dom_7_extension ); } else { addExtensionToCurrentChord( maj_7_extension );  } }

Pseudo code presented herein is meant to provide a high-informationcontent teaching tool for persons of ordinary skill in the art fromwhich may suitable adaptations, specializations, extensions andrealizations will be appreciated for particular deployments, codingenvironments, firmware and API definitions and procedural,objected-oriented, event-driven or other execution environment types.Pseudo code is not meant to limit embodiments in accordance with thepresent inventions to one particular coding style or expression of thefunctional decomposition and inventive techniques embodied therein.

Additional Embodiments and Deployments

FIG. 9 is a functional block diagram that further illustrates, inaddition to gesture capture and performance grading (previouslydescribed), optional communication of performance encodings and/orgrades as part of a game play or competition framework, social networkor content sharing facility in accordance with some embodiments of thepresent invention. As in the illustration of FIG. 4, an instance ofMagic Guitar application 450 executes on portable computing device 401.Note or chord soundings gestured (418) by the user musician at touchscreen/display 414 of portable computing device 401 are typically (atleast in some modes of operation) somewhat in correspondence withvisually presented note cues on touch screen/display 414 and are, inturn, captured and used to drive a digital synthesis (454) of acousticresponse of the synthetic guitar. In the illustrated configuration, inaddition to distinguishing note and chord sounding gestures so as toprovide a gesture stream 451 that drives synthesizer 454,capture/grading block 953 may evaluate quality of the note and/or chordsounding gesture expressions (e.g., timing of onset or strummingrelative to a score, sustenance of the selection gesture, expressedvibrato or other ornamentation) and, based thereon, award points forskill demonstration or game play modes. Performance encodings (either ascaptured gesture streams or encoded audio renderings) and well as pointcredits or grades may optionally be uploaded to a remote serviceplatform (e.g., by wireless communications 921) to facilitatecompetition and/or collaboration or to post or share in a socialnetworking environment.

FIG. 10 is a functional block diagram that illustrates capture, encodingand transmission of a gesture stream (or other) encoding correspondingto a user performance on a synthetic multi-string instrument (e.g., aninstance of Magic Guitar application 450 executes on portable computingdevice 401, recall FIGS. 4 and 9) together with receipt of such encoding(wirelessly communicated (1022) and acoustically rendered (1011)) on aremote device 1001. As in the prior illustrations, an instance of MagicGuitar application (here instance 450A) executes on the portablecomputing device (here remote device 1001). In some cases, modes orembodiments, note and/or chord soundings gestured (418) by the usermusician at touch screen/display 414 of portable computing device 401are captured and conveyed to remote device 1001, where they may be usedto drive a digital synthesis (454) of the synthetic guitar. Note thatwhile conveyance of the captured gesture stream from device 401 todevice 1001 may be supported in some cases, modes or embodiments,particularly where performance and capture of an accompaniment (atremote device 1001) is desired, in other cases, modes or embodiments, anencoding an audio rendering of the remotely captured performance may beconveyed and audibly rendered at remote device 1001. FIG. 11 is anetwork diagram that illustrates cooperation of exemplary devices inaccordance with some embodiments of the present invention.

While the invention(s) is (are) illustrated and described with referenceto various embodiments, it will be understood that these embodiments areillustrative and that the scope of the invention(s) is not limited tothem. Many variations, modifications, additions, and improvements arepossible. For example, while a synthetic guitar implementation has beenused as an illustrative example, variations on the techniques describedherein for other synthetic multi-string musical instruments (e.g.,guitars, violins, etc.) will be appreciated. Furthermore, while certainillustrative processing techniques have been described in the context ofcertain illustrative applications, persons of ordinary skill in the artwill recognize that it is straightforward to modify the describedtechniques to accommodate other suitable signal processing techniquesand effects.

Embodiments in accordance with the present invention may take the formof, and/or be provided as, a computer program product encoded in amachine-readable medium as instruction sequences and other functionalconstructs of software, which may in turn be executed in a computationalsystem (such as a iPhone handheld, mobile device or portable computingdevice) to perform methods described herein. In general, a machinereadable medium can include tangible articles that encode information ina form (e.g., as applications, source or object code, functionallydescriptive information, etc.) readable by a machine (e.g., a computer,computational facilities of a mobile device or portable computingdevice, etc.) as well as tangible storage incident to transmission ofthe information. A machine-readable medium may include, but is notlimited to, magnetic storage medium (e.g., disks and/or tape storage);optical storage medium (e.g., CD-ROM, DVD, etc.); magneto-opticalstorage medium; read only memory (ROM); random access memory (RAM);erasable programmable memory (e.g., EPROM and EEPROM); flash memory; orother types of medium suitable for storing electronic instructions,operation sequences, functionally descriptive information encodings,etc.

In general, plural instances may be provided for components, operationsor structures described herein as a single instance. Boundaries betweenvarious components, operations and data stores are somewhat arbitrary,and particular operations are illustrated in the context of specificillustrative configurations. Other allocations of functionality areenvisioned and may fall within the scope of the invention(s). Ingeneral, structures and functionality presented as separate componentsin the exemplary configurations may be implemented as a combinedstructure or component. Similarly, structures and functionalitypresented as a single component may be implemented as separatecomponents. These and other variations, modifications, additions, andimprovements may fall within the scope of the invention(s).

What is claimed is:
 1. A method of providing a synthetic instance of amulti-string musical instrument using a portable computing device, themethod comprising: capturing chord sounding gestures indicated by a useron a multi-touch sensitive display of the portable computing device,wherein the captured chord sounding gestures include, for particular andrespective chords so gestured, contact by multiple fingers of the user'shand and wherein the captured chord sounding gestures are characterizedby respective geometries of the multiple-finger contacts rather thansolely by positional registrations of individual finger contactsrelative to string or fret positions; distinguishing between respectiveof the multiple-finger contact geometries gestured by the user and basedthereon passing a stream of chord indicative inputs to a digitalsynthesis of the multi-string musical instrument executing on theportable computing device; and in correspondence with the stream ofcaptured chord indicative gestures, audibly rendering a performance onthe portable computing device using the digital synthesis.
 2. The methodof claim 1, wherein the chord sounding gesture capturing includesretrieving, in response to a multi-touch application programmerinterface (API) notification, a touch count and coordinates forrespective ones of the touches.
 3. The method of claim 2, wherein thedistinguishing includes, for two-touch contact gestures indicative of atwo-note chord: characterizing orientation of line traceable through thetouch coordinates and based thereon distinguishing between at least somesupported ones of the two-touch contact gestures.
 4. The method of claim3, wherein the distinguishing further includes characterizingfirst-dimension coordinates of a captured two-touch contact gesture todistinguish between major and minor chords.
 5. The method of claim 2,wherein the distinguishing includes, for three-touch contact gesturesindicative of a three-note chord: characterizing both shape defined bythe touch coordinates and an orientation thereof; and based on the shapeand orientation characterization, distinguishing between at least somesupported ones of the three-touch contact gestures.
 6. The method ofclaim 3, wherein the distinguishing further includes: for a three-touchcontact gesture characterized as an upward-pointed, triangulararrangement, indicating a G major chord; for a three-touch contactgesture characterized as an downward-pointed, triangular arrangement,indicating a D major chord; for a three-touch contact gesturecharacterized as a generally-linear arrangement, sloping upwardsleft-to-right, indicating a C major chord; and for a three-touch contactgesture characterized as a generally-linear arrangement, slopingdownwards left-to-right, indicating an F major chord.
 7. The method ofclaim 3, wherein the distinguishing further includes for four-touchcontact gestures indicative of a four-note chord: characterizing bothshape defined by coordinates of an initial three of the four touches andby an orientation thereof; further characterizing a positional relationbetween the characterized shape and a fourth of the four touches; basedon the shape and orientation characterization of the initial three (3)touch coordinates, initially distinguishing between at least somesupported major chords; and based on the characterized positionalrelation, distinguishing between major and dominant 7^(th) variants ofthe initially distinguished major chords.
 8. The method of claim 2,wherein the distinguishing includes characterizing second-dimensioncoordinates of a captured multi-touch contact gesture to distinguishbetween a root chord and one or more successive inversions thereof. 9.The method of claim 1, wherein the chord sounding gesture capturingincludes capture of both: a multi-touch chord selection gesture; and atraveling-touch, strum-type gesture.
 10. The method of claim 1, furthercomprising: providing a synthetic strum in accord with score-codedmeter, the synthetic strum releasing a constituent note or chordindications to the digital synthesis based on a then-current one of thecaptured note sounding gestures.
 11. The method of claim 1, wherein thedistinguishing is contextually constrained based at least in part on athen-current musical key.
 12. The method of claim 11, wherein thecurrent musical key is either user-selected or score-coded.
 13. Themethod of claim 1, further comprising: responsive to the captured chordsounding gestures, visually displaying strings of the multi-stringmusical instrument on the multi-touch sensitive display incorrespondence with the user's performance and at positions beneath oneor more of the respective finger contacts.
 14. The method of claim 13,wherein selection and positioning of the visually displayed strings aredynamically varied in correspondence with the captured chord soundinggestures.
 15. The method of claim 13, wherein fret positions and lateralextent of the visually displayed strings are dynamically varied incorrespondence with the captured chord sounding gestures.
 16. The methodof claim 1, further comprising: presenting the user of the syntheticmusical instrument with visual cues on the multi-touch sensitivedisplay, the presented visual cues indicative of temporally sequencedchord selections to be sounded by the user in accord with a musicalscore.
 17. The method of claim 16, wherein the chord-indicative visualcues are presented using a symbology evocative of shape and orientationof finger contacts that define a particular chord sounding gesture cued.18. The method of claim 16, wherein the presented visual cues include atleast some visual cues indicative of individual note selections.
 19. Themethod of claim 16, wherein notwithstanding multi-string nature of thesynthetic musical instrument, the presentation of the visual cues ispiano roll style, wherein individual visual cues travel toward pitchindicative positions along a laterally extended sounding zone of themulti-touch sensitive display.
 20. The method of claim 19, whereinlateral extent of the sounding zone is dynamically allocated so thatpitch range of available notes and spacing between note selectivepositions distributes a current contextually relevant set of soundingpositions along the available lateral extent of a multi-touch sensitivedisplay.
 21. The method of claim 16, further comprising: determiningcorrespondence of respective captured chord sounding gestures with thevisually cued chord selections; and grading the user's performance atleast in part based on the determined correspondences, wherein thedetermined correspondences include (i) a measure of temporalcorrespondence of a particular chord sounding gesture with arrival of avisual cue in the sounding zone and (ii) a measure correspondence of thechord sounded with the visual cue.
 22. The method of claim 1, whereinthe stream of chord indicative inputs includes constituent pluralitiesof note selection inputs consistent with a temporal sequence of thecaptured and distinguished chord sounding gestures.
 23. The method ofclaim 1, wherein the digital synthesis includes a sample-based synthesisof constituent notes of the captured chord.
 24. The method of claim 1,wherein the digital synthesis includes a sample-based synthesis of thecaptured chord.
 25. The method of claim 1, wherein the audible renderingincludes: modeling acoustic response for the multi-string instrument;and driving the modeled acoustic response with inputs corresponding tothe captured chord sounding gestures.
 26. The method of claim 16,wherein the synthetic multi-string instrument is a guitar, and whereinthe visual cues travel across the multi-touch sensitive display andrepresent, in one dimension of the multi-touch sensitive display,desired finger contacts along a fretless single-string analog of themulti-string instrument in accordance with notes of the score and, in asecond dimension generally orthogonal to the first, temporal sequencingof the desired finger contacts paced in accord with a current tempo. 27.The method of claim 1, further comprising: presenting on the multi-touchsensitive display a lesson plan of exercises, wherein the captured chordselection gestures correspond to performance by the user of a particularone of the exercises; and advancing the user to a next exercise of thelesson plan based on a grading of the user's performance of theparticular exercise.
 28. The method of claim 1, wherein the portablecomputing device includes a communications interface, the method furthercomprising, transmitting an encoded stream of the note selectiongestures via the communications interface for rendering of theperformance on a remote device.
 29. The method of claim 28, furthercomprising: geocoding the transmitted gesture stream; and displaying ageographic origin for, and in correspondence with audible rendering of,another user's performance encoded as another stream of notes soundinggestures received via the communications interface directly orindirectly from a remote device.
 30. The method of claim 1, wherein theportable computing device is selected from the group of: a compute pad;a game controller; a personal digital assistant or book reader; and amobile phone or media player.
 31. A computer program product encoded inone or more non-transitory media, the computer program product includinginstructions executable on a processor of the portable computing deviceto cause the portable computing device to perform the method of claim 1.32. The computer program product of claim 31, wherein the one or morenon-transitory media are readable by the portable computing device orare readable in a course of conveying the computer program product tothe portable computing device.
 33. An apparatus comprising: a portablecomputing device having a multi-touch sensitive display; and machinereadable code executable on the portable computing device to implement asynthetic musical instrument, the machine readable code includinginstructions executable to capture chord sounding gestures indicated bya user on the multi-touch sensitive display, wherein the capture ofchord sounding gestures includes, for particular and respective chordsso gestured, registration of contact by multiple fingers of the user'shand and distinguishing between respective geometries of themultiple-finger contacts, wherein the captured chord sounding gesturesare characterized by the respective multiple-finger contact geometriesrather than solely by positional registrations of individual fingercontacts relative to string or fret positions, the machine readable codestill further executable to audibly render the user's performance on theportable computing device using, as an input to a digital synthesis ofthe synthetic musical instrument executing on the portable computingdevice, a gesture stream that includes the captured chord soundinggestures, wherein the gesture stream, and not the musical score itself,drives the digital synthesis.
 34. The apparatus of claim 33, embodied asone or more of a compute pad, a game controller, a handheld mobiledevice, a mobile phone, a personal digital assistant, a media player anda book reader.
 35. A computer program product encoded in non-transitorymedia and including instructions executable to implement a syntheticmulti-string musical instrument on a portable computing device having amulti-touch display interface, the computer program product encoding andcomprising: instructions executable by the portable computing device tocapture chord sounding gestures indicated by a user on the multi-touchsensitive display, wherein the captured chord sounding gestures include,for particular and respective chords so gestured, contact by multiplefingers of the user's hand and wherein the captured chord soundinggestures are characterized by respective geometries of themultiple-finger contacts rather than solely by positional registrationsof individual finger contacts relative to string or fret positions;instructions executable by the portable computing device to distinguishbetween respective of the multiple-finger contact geometries gestured bythe user and based thereon to pass a stream of chord indicative inputsto a digital synthesis of the multi-string musical instrument executingon the portable computing device; and instructions executable by theportable computing device as the digital synthesis to audibly render aperformance in correspondence with the stream of captured chordindicative gestures.
 36. The computer program product of claim 35,wherein the instructions executable by the portable computing device todistinguish between respective of the multiple-finger contactgeometries, include instruction sequences to distinguish two-, three-and four-touch gestures.
 37. The computer program product of claim 35,wherein the instructions executable by the portable computing device tocapture chord sounding gestures capture both multi-touch chord selectiongestures and a traveling-touch, strum-type gesture.
 38. The computerprogram product of claim 35, further comprising: instructions executableto determine correspondence of respective captured chord soundinggestures with the chord selections visually cued in the sounding zoneand to grade the user's performance based on the determinedcorrespondences.
 39. The computer program product of claim 35, whereinthe non-transitory media are readable by the portable computing deviceor are readable in a course of conveying the computer program product tothe portable computing device.
 40. A method comprising: using a portablecomputing device as a multi-string synthetic musical instrument;presenting a user of the synthetic musical instrument with visual cueson a multi-touch sensitive display of the portable computing device, thepresented visual cues indicative of temporally sequenced note selectionsto be sounded by the user in accord with a musical score, whereinnotwithstanding multi-string nature of the synthetic musical instrument,the presentation of the visual cues is piano roll style, whereinindividual visual cues travel toward pitch indicative positions along alaterally extended sounding zone of the multi-touch sensitive display,and wherein at least one of the presented visual cues indicates a stringbend performance effect to be sounded by the user in accord with themusical score; capturing note sounding gestures indicated by the userusing the multi-touch sensitive display; and audibly rendering theperformance on the portable computing device using the captured gesturestream as an input to a digital synthesis of the synthetic musicalinstrument executing on the portable computing device, wherein thecaptured gesture stream, and not the musical score itself, drives thedigital synthesis.
 41. The method of claim 40, wherein the captured notesounding gestures include finger contacts at pitch selective positionsalong the laterally extended sounding zone.
 42. The method of claim 40,wherein the captured note sounding gestures for the string bendperformance effect include finger contact and in-contact travel on themulti-touch sensitive display in a direction generally orthogonal tolateral extent of the sounding zone.
 43. The method of claim 40, whereinthe audible rendering in correspondence with a captured string bendindicative gesture varies pitch from an initial pitch to a pitchcorresponding to a note selection coded in the musical score.
 44. Themethod of claim 40, wherein the audible rendering in correspondence witha captured string bend indicative gesture varies pitch from a pitchcorresponding to a note selection coded in the musical score.
 45. Themethod of claim 40, wherein lateral extent of the note sounding zone isdynamically allocated so that pitch range of available notes and spacingbetween note selective positions distributes a current contextuallyrelevant set of note sounding positions along the available lateralextent of a multi-touch sensitive display.
 46. The method of claim 40,wherein the captured note sounding gestures include, for a particularchord visually cued in accordance with the musical score, contact bymultiple fingers of the user's hand.
 47. The method of claim 46, furthercomprising: distinguishing between plural multi-finger contactgeometries as chord indicative gestures; and in correspondence with acaptured chord indicative gesture, audibly rendering a correspondingmulti-string chord using the digital synthesis.
 48. The method of claim40, wherein at least one of the presented visual cues indicates avibrato performance effect in accord with the musical score; and furthercomprising capturing and including a vibrato indicative gesture in thegesture stream supplied as input to the digital synthesis of thesynthetic string instrument executing on the portable computing device.49. The method of claim 48, further comprising: capturing the vibratoindicative gesture using an on-board accelerometer of the portablecomputing device.
 50. The method of claim 40, further comprising:determining correspondence of respective captured note sounding gestureswith the note selections visually cued in the sounding zone; and gradingthe user's performance based on the determined correspondences.
 51. Themethod of claim 50, wherein the determined correspondences include: ameasure of temporal correspondence of a particular note sounding gesturewith arrival of a visual cue in the sounding zone; and a measure of noteselection correspondence of the particular note sounding gesture withthe visual cue.
 52. The method of claim 50, wherein the determinedcorrespondences include: a measure of temporal correspondence of thestring bend performance effect with arrival of the corresponding visualcue in the sounding zone.
 53. The method of claim 40, wherein thesynthetic multi-string instrument is a guitar, and wherein the visualcues travel across the multi-touch sensitive display and represent, inone dimension of the multi-touch sensitive display, desired fingercontacts along a fretless single-string analog of the multi-stringinstrument in accordance with notes of the score and, in a seconddimension generally orthogonal to the first, temporal sequencing of thedesired finger contacts paced in accord with a current tempo.
 54. Themethod of claim 53, wherein the sounding zone corresponds generally to agenerally linear display feature on the multi-touch sensitive displaytoward or across which the visual cues travel.
 55. The method of claim40, wherein the captured note sounding gestures are indicative of bothstring excitation and pitch selection for the excited string.
 56. Themethod of claim 40, further comprising: presenting on the multi-touchsensitive display a lesson plan of exercises, wherein the captured noteselection gestures correspond to performance by the user of a particularone of the exercises; and advancing the user to a next exercise of thelesson plan based on a grading of the user's performance of theparticular exercise.
 57. The method of claim 40, wherein the portablecomputing device includes a communications interface, the method furthercomprising, transmitting an encoded stream of the note selectiongestures via the communications interface for rendering of theperformance on a remote device.
 58. The method of claim 57, furthercomprising: geocoding the transmitted gesture stream; and displaying ageographic origin for, and in correspondence with audible rendering of,another user's performance encoded as another stream of notes soundinggestures received via the communications interface directly orindirectly from a remote device.
 59. The method of claim 40, wherein theaudible rendering includes: modeling acoustic response for themulti-string instrument; and driving the modeled acoustic response withinputs corresponding to the captured note sounding gestures.
 60. Themethod of claim 40, wherein the portable computing device is selectedfrom the group of: a compute pad; a game controller; a personal digitalassistant or book reader; and a mobile phone or media player.
 61. Acomputer program product encoded in one or more non-transitory media,the computer program product including instructions executable on aprocessor of the portable computing device to cause the portablecomputing device to perform the method of claim
 40. 62. The computerprogram product of claim 61, wherein the one or more non-transitorymedia are readable by the portable computing device or are readable in acourse of conveying the computer program product to the portablecomputing device.
 63. An apparatus comprising: a portable computingdevice having a multi-touch display interface; and machine readable codeexecutable on the portable computing device to implement a syntheticmusical instrument, the machine readable code including instructionsexecutable to present a user of the synthetic musical instrument withvisual cues on the multi-touch sensitive display, the presented visualcues indicative of temporally sequenced note selections to be sounded bythe user in accord with a musical score, wherein notwithstandingmulti-string nature of the synthetic musical instrument, thepresentation of the visual cues is piano roll style, wherein individualones of the visual cues travel toward pitch indicative positions along alaterally extended sounding zone of the multi-touch sensitive display,and wherein at least one of the presented visual cues indicates a stringbend performance effect to be sounded by the user in accord with themusical score; the machine readable code further executable to capturenote sounding gestures indicated by the user using the multi-touchsensitive display and to audibly render the performance on the portablecomputing device using the captured gesture stream as an input to adigital synthesis of the synthetic musical instrument executing on theportable computing device, wherein the captured gesture stream, and notthe musical score itself, drives the digital synthesis.
 64. Theapparatus of claim 63, the machine readable code further executable to,in correspondence with a captured string bend indicative gesture, varyan audibly rendered pitch to or from a pitch corresponding to a noteselection coded in the musical score.
 65. The apparatus of claim 63, themachine readable code further executable to distinguishing betweenplural multi-finger contact geometries as chord indicative gestures and,in correspondence with a captured chord indicative gesture, audiblyrender a corresponding multi-string chord using the digital synthesis.66. The apparatus of claim 63, embodied as one or more of a compute pad,a game controller, a handheld mobile device, a mobile phone, a personaldigital assistant, a media player and a book reader.
 67. A computerprogram product encoded in non-transitory media and includinginstructions executable to implement a synthetic multi-string musicalinstrument on a portable computing device having a multi-touch displayinterface, the computer program product encoding and comprising:instructions executable by the portable computing device to present auser of the synthetic musical instrument with visual cues on themulti-touch sensitive display, the presented visual cues indicative oftemporally sequenced note selections to be sounded by the user in accordwith a musical score, wherein notwithstanding multi-string nature of thesynthetic musical instrument, the presentation of the visual cues ispiano roll style, wherein individual ones of the visual cues traveltoward pitch indicative positions along a laterally extended soundingzone of the multi-touch sensitive display, and wherein at least one ofthe presented visual cues indicates a string bend performance effect tobe sounded by the user in accord with the musical score; instructionsexecutable by the portable computing device to capture note soundinggestures indicated by the user using the multi-touch sensitive display;and instructions executable by the portable computing device to audiblyrender the performance on the portable computing device using thecaptured gesture stream as an input to a digital synthesis of thesynthetic musical instrument executing on the portable computing device,wherein the captured gesture stream, and not the musical score itself,drives the digital synthesis.
 68. The computer program product of claim63, further comprising: instructions executable to determinecorrespondence of respective captured note sounding gestures with thenote selections visually cued in the sounding zone and to grade theuser's performance based on the determined correspondences.
 69. Thecomputer program product of claim 63, wherein the non-transitory mediaare readable by the portable computing device or are readable in acourse of conveying the computer program product to the portablecomputing device.